Evaluating Large-Scale Integrated Care Projects: The Development of a Protocol for a Mixed Methods Realist Evaluation Study in Belgium

Background: The twelve Integrated Care Program pilot projects (ICPs) created by the government plan ‘Integrated Care for Better Health’ aim to achieve four outcome types (the Quadruple Aim) for people with chronic diseases in Belgium: improved population health, improved patient and provider experiences and improved cost efficiency. The aim of this article is to present the development of a mixed methods realist evaluation of this large-scale, whole system change programme. Methods: A scientific team was commissioned to co-design and implement an evaluation protocol in close collaboration with the government, the ICPs and several other involved stakeholders. Results: A protocol for a mixed methods realist evaluation was developed to gain insights into the mechanisms that foster successful results in ICPs. The qualitative evaluation proposed will be based on the document analysis of yearly ICP progress reports, selected case studies and focus group interviews with stakeholders. Processes and outcomes of all the projects will be monitored using indicators based on administrative data on population health and the quality and costs of care. A yearly survey will be organized to collect data on patient-reported outcomes and experiences and on provider-reported measures of inter-professional collaboration and proper wellbeing. Using both quantitative and qualitative data, we will develop theories about the mechanisms and the associated contextual factors that lead to integrated care and the Quadruple Aim outcomes. Discussion: The objective of this study is to deliver policy recommendations on strategies and best practices to improve care integration in Belgium and to implement a sustainable monitoring system that serves both policy makers and the stakeholders within the ICPs. Some challenges due to the large scale of the project and the multiple stakeholders involved may impede the successful implementation of this proposal.FAITH.be - Federated consortium for Appraisal of Integrated care Teams in Health in Belgiu

How to evaluate a Nationwide Whole System Change in Health Care towards Integrated Care for people with Chronic Conditions? Part I: the co-construction of an evaluation framework

peer reviewe

Scientific evaluation and support to the plan “Integrated Care for Better Health” - Final report

FAITH.be - Federated consortium for Appraisal of Integrated care Teams in Health in Belgiu

Scientific evaluation and support to the plan "Integrated care for better health" - Executive summary

Realist and developmental evaluation approach The policy plan “integrated care for better health”, and more specifically its first action line, the implementation of 12 integrated care pilot projects (ICPs) since 2018, has provided very promising results over the two first years. This is an ongoing process: the endeavour should be sustained by different stakeholders acting at different levels in order to yield fruitful results in the near future. The opposite would represent a considerable loss of investment and of knowledge “in the making” to integrate care in Belgium. The scientific team FAITH.be, in charge of evaluation of the Plan, was able to (1) create and apply to iCPs a monitoring framework for development of integrated care based on a whole population management within a geographical area (see p 30 to 48 of executive summary), (2) identify several promising practices (see in main report and specific annex) that should be further developed, and (3) identify specific conditions that can help reaching the overall goals of the Plan. This work of the realist and developmental evaluation initiated by FAITH.be (including triple aim + 2 indicators) should be used to strengthen the process of integrated care development. Indeed, the authorities should acknowledge the complex nature of the changes taking place via this ambitious system reform, involving many stakeholders at many levels who do not have a definitive vision about the choices to be made or how to proceed. All the involved stakeholders can contribute to knowledge production about these complex processes: this is a key goal of the policy Plan. This part summarizes the main lessons learned in the form of “recommendations” addressed to the policy makers at federal and federated level, and to the stakeholders involved in ICP development. More information about why these recommendations are developed, how they were identified and concrete examples can be found in the body of the executive summary, the full report and it’s annexes. List of concrete recommendations for federal and federated authorities: Develop further methods, tools and spaces to share the knowledge on integrated care and translate it for decision making: • Give a priority to specific moments with key stakeholders (every 6 months or annually) where experience, and knowledge on integrated care can be shared and translate into adapted decisions • Appoint (from within the administration or outside) skilled people to promote the use, and develop further, the monitoring framework proposed by FAITH for learning and strengthening the development of integrated care based on a whole population management within a geographical area (see p 30 to 48 of executive summary). For actions to improve care integration, this means adopting a “whole local health system” approach. For triple aim + 2 indicators, including health care utilization and cost, this means triangulating information from transition between strata, evolution over time of triple aim indicators and evolution over time of context (population and health care utilization). • Continue the process of creation of a dashboard as a tool for learning and development of integrated care. Administration (IAC) need to facilitate the process by coordinating (1) the work of data flow coming from various database (IMA for administrative and included population, , BelRAI, MZG, PROMS-PREMS), (2) the development of the data warehouse functions of healthdata (including linkage between database for administrative and included population), (3) the technical work of designing the dashboard with visual analytics for SAS or other software. This can build on the existing work realized by FAITH.be on (1) ethical and privacy procedures and (2) methodology for context , stratification and triple aim + 2 indicators analysis. Specifically, organize a yearly PROMS&PREMS survey and the professional survey, in collaboration with the ICPs. In order to collect sufficient data - ICP coordinators and providers should be fully engaged and thus should gain ‘ownership’ of the evaluation procedure. Moreover, involvement of the sickness funds in the data collection procedure, complementary to the ICP procedure seems promising. However, the procedure in itself should be evaluated, especially the identification procedure with ID or ITSME. • Support ICPs knowledge management (with a dedicated support team) through: o the continuation of community of practices that allow for experience sharing, o individualized coaching of projects since they have very different needs and expertise o Keep documenting and sharing promising / good practices. Enhance change management done by ICPs via the following elements: • Better support patients organizations. Indeed, these are currently overwhelmed by demands but have little means to respond to these demands. • Develop official national communication campaign about integrated care, emanating from the federal authorities and aimed at the broader public, including care providers, patients, and other potential partners could help to create momentum towards change. • Promote training in integrated care development (through courses or coaching) • Promote training for patient centred and interdisciplinary care (p57; 204; 238). Give a priority through e-health roadmap implementation to further develop IT tools for care integration – exchange of information: • Develop integrated patient file between hospital and ambulatory care, and between different professions, so different providers can work on the same files while respecting privacy issues (See p.64 for a description of the modalities expected by the ICP’s and p.238 for conceptual reflections). Develop further the role of the patient in managing his own health record • Improve user friendliness of inclusion application, so as to allow more patient flagging/monitoring (see p. 59) • BelRAI: solve operational problems and align the federal and federated BelRAI policies (see p.61) • reduce the administrative burden linked with IT tools for providers Provide a safe learning environment for pilot project development and experimentations: • Provide a long term perspective for ICPs as a 4 years’ time frame is too short to make the paradigm shift towards integrated care for better health in a whole region, involving many stakeholders (see p232). • The performance-based assessment of ICPs needs to contribute to better understanding of local condition for integrated care development. The use of whole local health system and whole population framework by ICPs, administrations and authorities should be further encouraged. This should be a priority compared to the use of indicators for summative approach, especially if it leads to efficiency gains distribution. Indeed, a narrow set of “measurable” indicators represents a high risk for the whole policy plan: ICPs might forget the overall goal of testing innovative care integration modalities by focusing first and foremost on the narrow goal of performing well regarding a limited set of indicators. • Further look at legal possibilities to allow more freedom to experiment with innovative ways of organizing care – especially regarding the use of efficiency gains at ICP level (annex 7). The federated authorities of Flanders should also allow more freedom (regelluwte; see p.69). • The future financing of the projects needs to be decided, both in duration and the amount per project, as the lack of clarity regarding this issue is a source of insecurity for engaged partners (see p70-74). A solution could be to combine dedicated, stable funding with a rewarding system based on efficiency gains. • All of these requirement should help ICPs to attract a strong team of coordinators, with a mix of the required competences (see p31; 54; 95). Develop an (evolving) integrated vision document of the health and social care system in Belgium (p236): • The plan “Integrated Care for Better Health” should be articulated with hospital networks, territorial approach for primary care (e.g eerstelijnzorgzones and SISD) and other concurrent policy reforms that impact the healthcare system’s organization • Develop an overall change management strategy for care integration in Belgium, to raise awareness and show the added value of care integration for both professionals and the broad public. List of concrete recommendations for ICPs: Apply further monitoring framework for evidence-based decision in the development of integrated care and PDSA implementation by ICP governance: • Implement further actions to reach large scale, and continue documenting it (using, for example, the template developed for promising practices). • Learn further how to use triple aim+2 indicators and particularly population healthcare utilization indicators to develop population management approach and adapt actions based on changes observed at loco-regional level. • Governance and integration management should learn further o To triangulate different sources of data and type of information (from context, strata, cost) and to interpret their evolution over time, paying special attention to equity issues o To use that analysis for regular (yearly) actions adaptation and ultimately resources re-allocation within the local health system. • Recruit partners with some expertise in integrated care evaluation. • Use the tools and templates provided by FAITH.be (annual report, reporting of significant events and identification of promising practices thanks to repeated exchanges between ICPs) Develop further change management (see p.101): • Invest further in communication (communication strategy to explain the project, goals and actions): o Communication within the consortium: to help the partners stay up to date and to help them in their tasks (newsletters, project ambassadors, study days, training sessions, practical guidelines…). o Communication aimed at the broader public, including patients, care providers and other potential partners (website, social media, flyers, ambassadors…) • Invest to keep the partners engaged, and to attract new partners through a customized action implementation strategy • Further develop the required skills among professionals, patients and informal caregivers, e.g. via the organization of training sessions and continuing education • Further efforts are needed to facilitate the participation of patients and informal caregivers in decision making List of promising practices at ICP level: The promising practices identified by FAITH.be are actions or ways of working that can help ICPs achieve their goals. These promising practices are developed following a template highlighting key messages (what, how, why…) and concrete examples emanating from the projects. ICPs can find some inspiration in these practices; they should be encouraged to keep identifying and circulating other promising practices. These promising practices will be grouped and translated in a specific deliverable to be circulated broadly. Promising practices regarding ICP governance: Collaboration and collective decision making (p92) • Governance structure development - how to recruit partners? Formalizing the collaboration • Support stakeholders representatives - Transparent documentation of decision making processes • Shared leadership for sustainability • Support patient representative involvement in governance • Building collaborative relationships - Invest in networking Change management (p 101) • Build on pre-existing federative structures to recruit partners • Develop effective communication to promote the ICP towards patients and care providers • Develop a shared vision and objectives • Create a win-win situation to engage partners • Interdisciplinary bottom-up approach for action development • Start implementing actions • Effective communication to clarify the role of each partner within an action Development of a quality culture (p111) • Learning communities/communities of practice to learn from experience, make adaptation and share lessons Promising practices for care integration: Promising practice for care integration are implemented by the various ICPs. They provide valuable information to develop context-sensitive strategies to improve care integration by acting at the differents parts of the whole local health system. They illustrate the following strategies (see details in specific annex) Primary care development integrated with specific health program within a neighbourhood approach • Zorgcoördinatie en een meer gestructureerde eerstelijn in buurtteams – Zorgzaam Leuven (ZZL) Goal oriented case management by primary care team or network • Plan de besoins individualisé - PBI (Relian) • Référent de proximité (BOOST) Primary care network development through sharing tasks • Dépistage du diabète en pharmacie (CHRONILUX) • Diagnostic du SAOS au niveau de la première ligne de soins (Chronicopole) Enhancement of integration between hospital and ambulatory care • Zorgpad hartfalen – De Brug • Unités Thérapeutiques Intégrées Locales- UTIL’s (PACT) • Dagziekenhuis Parkinson- EMPACT Specific functions for complex situation case management • Case manager (RéSINamur) • Dementiecoach – Geïntegreerde zorg Waasland Overall patient-centred network integration through patient information • Zorg-ID - De Koepel Quasi-experimental approach in evaluation: Future monitoring of Triple Aim 2+ indicators As to the monitoring tools • We advise to continue to use the tools that Faith.be has set up allowing for a long term evaluation of “triple aim 2+” indicators (in the international literature commonly known as ‘Quadruple Aim’ indicators). The evaluation tools can be used for formative evaluation in support of the projects by means of the dashboard and ‘distal’ summative evaluation in the future, at the appropriate moment • For the moment, the healthdata.be platform includes data from the IMA data warehouse and the flagging of people included in an ICP. In the future, data from other sources (BELRAI, Hospital Data, and General Practitioner Data) could be included. However, this will induce an additional workload and will enhance complexity in procedures, management, analyses and interpretation. It is not clear yet what will be the ‘investment/benefit’ ratio. • Because of the nature of the Integreo projects, the nature of the data and the method of the data collection, it is likely that a sense-making evaluation of the Triple Aim 2+ indicators will be in need of repeated data collections over several years. Data collections and analyses should preferable happen on a yearly basis. A team for quasi-experimental design and summative evaluation will remain necessary in the future • Future evaluation will require ongoing supervision and teamwork by some kind of ‘Integreo Evaluation Team’ This team will have following tasks: o Managing all legal and ethical aspects: e.g. licensing of survey tools, future agreements of the Data Protection Authority and Ethical Committee, o Managing and cleaning the data from different sources o Merging the data at the healthdata.be platform o Organizing the data collections of PROMS&PREMS survey and professional survey o Performing the statistical analyses o Support the ICP’s with the interpretation of the results • Considering the composition of this team: o It should at least include specialists in legal and ethical matters, data managers and statistician(s). o It is recommendable that Healthdata.be and the Inter Mutualistic Agency are directly engaged in this team. o Involvement of a specialist in health systems research and a person with clinical knowledge is recommendable for the interpretation of the results Considering the involvement of the ICPs (in their proper evaluation) • ICPs (including professionals and patients) are the primary beneficiaries of the intended quality improvement. • Statistically significant change is in need of size and time. Therefore, • If auto-evaluation by the projects will be the cornerstone of the future evaluation, direct involvement of the ICPs in the management of the procedures, the statistical analyses of the results and their interpretation is highly recommendable. • Therefore, we recommend that in the future, the ICPs will designate a key person for analysis and auto-evaluation who joins a collaborative structure to foster co-creation between the evaluation team and the ICPs. As to the professional survey • As these pilot results are promising, repeated, yearly surveys may reveal the status and trends on professional well-being of health care workers in the ICPs • For 12 ICPs, it approximately takes 1 Full Time Equivalent to perform the data collection (which means 1 month data collection per ICP per year) • ICP involvement in the data collection is desirable. 1 dedicated person per ICP who has connections with the local professionals and spends 1 month a year collecting the data seems a better choice than 1 central person who is fully and only committed to collect those data for all ICPs. As to the PROMS&PREMS • The first data collection should be considered as a pilot • Since contracts have ended at June 30th 2020, legal and ethical approvals should be reconsidered as well as the team who will lead the data collection and perform the analysis and interpretation of results • Involvement of the sickness funds in the data collection procedure, complementary to the ICP procedure seems promising. However, the procedure in itself should be evaluated, especially the identification procedure with ID or ITSME. As to both surveys • if valid results are desired, then the data collections have to happen scientifically sound and thus organized as a campaign ‘lege artis’, with overall supervision and appropriate investments in budget and manpowe

Clofazimine Analogs with Efficacy against Experimental Tuberculosis and Reduced Potential for Accumulation▿

The global tuberculosis crisis urgently demands new, efficacious, orally available drugs with the potential to shorten and simplify the long and complex treatments for drug-sensitive and drug-resistant disease. Clofazimine, a riminophenazine used for many years to treat leprosy, demonstrates efficacy in animal models of tuberculosis via a novel mode of action. However, clofazimine's physicochemical and pharmacokinetic properties contribute to side effects that limit its use; in particular, an extremely long half-life and propensity for tissue accumulation together with clofazimine's dye properties leads to unwelcome skin discoloration. We recently conducted a systematic structure-activity study of more than 500 riminophenazine analogs for anti-Mycobacterium tuberculosis activity. We describe here the characteristics of 12 prioritized compounds in more detail. The new riminophenazine analogs demonstrated enhanced in vitro activity compared to clofazimine against replicating M. tuberculosis H37Rv, as well as panels of drug-sensitive and drug-resistant clinical isolates. The new compounds demonstrate at least equivalent activity compared to clofazimine against intracellular M. tuberculosis and, in addition, most of them were active against nonreplicating M. tuberculosis. Eleven of these more water-soluble riminophenazine analogs possess shorter half-lives than clofazimine when dosed orally to mice, suggesting that they may accumulate less. Most importantly, the nine compounds that progressed to efficacy testing demonstrated inhibition of bacterial growth in the lungs that is superior to the activity of an equivalent dose of clofazimine when administered orally for 20 days in a murine model of acute tuberculosis. The efficacy of these compounds, along with their decreased potential for accumulation and therefore perhaps also for tissue discoloration, warrants further study